Allow to export dashboards from Serverless environments #3007

mrodm · 2025-10-20T14:30:40Z

This PR allows to export dashboards in Serverless environments (elastic-package export dashboards command).

For Serverless environments, a new set of APIs from Elastic Package Manager have been used:

Get the current packages installed: https://www.elastic.co/docs/api/doc/kibana/operation/operation-get-fleet-epm-packages-installed
Get the information about those packages (containing asset IDs): https://www.elastic.co/docs/api/doc/kibana/operation/operation-get-fleet-epm-packages-pkgname-pkgversion
Get the actual description of those assets via https://www.elastic.co/docs/api/doc/kibana/operation/operation-post-fleet-epm-bulk-assets

If this command is used in local stacks, the same behavior is kept. Given that, all dashboards are shown to select which ones to export.

Same changes are applied to the elastic-package edit dashboards command.

Author's checklist

Check elastic-package export dashboards in Serverless
Check elastic-package export dashboards in local stacks (same behavior).
Check elastic-package edit dashboards in Serverless
Check elastic-package edit dashboards in local stacks (same behavior).

Both commands require the same changes to list dashboards.

Example of usage

First step all installed packages are shown:

 $ elastic-package -C ../integrations-main/packages/nginx export dashboards 
Export Kibana dashboards
Which packages would you like to export dashboards from? (nginx-2.3.2)
  Working on a new dashboard (show all available dashboards)                 
  elastic_package_registry-0.3.1
  fleet_server-1.6.0            
> nginx-2.3.2                   
  synthetics-1.4.2              
                                

enter continue • ctrl+c cancel

If the user selects a package, it will be shown all the dashboards related to that package (e.g. nginx):

 $ elastic-package -C ../integrations-main/packages/nginx export dashboards 
Export Kibana dashboards
? Which packages would you like to export dashboards from?: nginx-2.3.2
Which dashboards would you like to export?
  Use ↑↓ to navigate, space to toggle selection, enter to confirm
> [ ] [Logs Nginx] Access and error logs (ID: nginx-046212a0-a2a1-11e7-928f-5dbe6f6f5519, Type: dashboard)
  [ ] [Logs Nginx] Overview (ID: nginx-55a9e6e0-a29e-11e7-928f-5dbe6f6f5519, Type: dashboard)             
  [ ] [Metrics Nginx] Overview (ID: nginx-023d2930-f1a5-11e7-a9ef-93c69af7b129, Type: dashboard)          

enter continue • ctrl+c cancel

If the user is developing a new dashboard (and therefore not related to any package), they should choose New dashboard option, and all the dashboards available in the Serverless instance will be shown:

 $ elastic-package -C ../integrations-main/packages/nginx export dashboards 
Export Kibana dashboards
Which packages would you like to export dashboards from? (nginx-2.3.2)
> Working on a new dashboard (show all available dashboards)                 
  elastic_package_registry-0.3.1
  fleet_server-1.6.0            
  nginx-2.3.2                   
  synthetics-1.4.2              
                                

enter continue • ctrl+c cancel

# After choosing "New dashboard" option

 $ epd -C ../integrations-main/packages/nginx export dashboards 
Export Kibana dashboards
? Which packages would you like to export dashboards from?: Working on a new dashboard (show all available dashboards)
Which dashboards would you like to export?
  Use ↑↓ to navigate, space to toggle selection, enter to confirm
> [ ] [EPR Metrics] Overview (ID: elastic_package_registry-313c2700-099b-11ed-91b6-3b1f9c2b2771)
  [ ] [Logs Nginx] Access and error logs (ID: nginx-046212a0-a2a1-11e7-928f-5dbe6f6f5519)       
  [ ] [Logs Nginx] Overview (ID: nginx-55a9e6e0-a29e-11e7-928f-5dbe6f6f5519)                    
  [ ] [Metrics Nginx] Overview (ID: nginx-023d2930-f1a5-11e7-a9ef-93c69af7b129)                 

enter continue • ctrl+c cancel

How to test this PR locally

Start a new elastic-package with the serverless provider as detailed in https://github.com/elastic/elastic-package/blob/main/docs/howto/use_serverless_stack.md#the-serverless-stack-provider
Install a package and run export dashboards

cd /path/to/package
elastic-package install -v

elastic-package export dashboards -v
elastic-package export dashboards -v --id <some_dashboard_id>

elastic-package edit dashboards -v
elastic-package edit dashboards -v --id <some_dashboard_id>

Extract function to select dashboard IDs

mrodm · 2025-10-20T14:46:52Z

internal/packages/assets.go

+	// These fields are not expected to be part of responses from APIs
+	DataStream string `json:"-"`
+	SourcePath string `json:"-"`


I could not find any usage of these two fields in APIs.
It looks like that they are used in test asset and they are filled programmatically.

elastic-package/internal/packages/assets.go

Line 139 in fc03bdf

DataStream: dsManifest.Name,

elastic-package/internal/packages/assets.go

Line 171 in fc03bdf

DataStream: dsManifest.Name,

elastic-package/internal/packages/assets.go

Line 207 in fc03bdf

SourcePath: assetPath,

If the json:"-" tag is not added to the fields, queries with this struct fail with the error of those fields being not available/undefined.

mrodm · 2025-10-20T16:46:18Z

internal/kibana/dashboards.go

+func (c *Client) exportAllDashboards(ctx context.Context) ([]common.MapStr, error) {
+	logger.Debug("Export dashboards using the Kibana Saved Objects Export API")
+
+	request := ExportSavedObjectsRequest{
+		ExcludeExportDetails:  true,
+		IncludeReferencesDeep: false,
+		Type:                  "dashboard",
+	}
+
+	return c.ExportSavedObjects(ctx, request)
+}


This method would return all the dashboards available in the Kibana Instance.

According to the documentation, the number of assets that will be exported depend on savedObjects.maxImportExportSize setting (link):

NOTE: The savedObjects.maxImportExportSize configuration setting limits the number of saved objects which may be exported.

By default that setting is 10000 elements (savedObjects.maxImportExportSize setting):

The maximum count of saved objects that can be imported or exported. This setting exists to prevent the Kibana server from running out of memory when handling large numbers of saved objects. It is recommended to only raise this setting if you are confident your server can hold this many objects in memory. Default: 10000

mrodm · 2025-10-21T07:53:14Z

/test

mrodm · 2025-10-21T13:30:48Z

internal/kibana/savedobjects.go

+	// Even if no error is returned, we need to check if the import was successful.
+	if !resp.Success {
+		return fmt.Errorf("importing %s %s was not successful", savedObjectType, id)
+	}
+


Added a new check to ensure that the saved object has been imported successfully:
https://www.elastic.co/docs/api/doc/kibana/operation/operation-post-saved-objects-import

success: Indicates when the import was successfully completed. When set to false, some objects may not have been created.

elasticmachine · 2025-10-21T13:43:22Z

💛 Build succeeded, but was flaky

Buildkite Build
Commit: 9cb72c3

Failed CI Steps

:go: Integration test: stack-command-oldest

History

💔 Build #6393 failed a8b59fb
💔 Build #6387 failed 6542265
💔 Build #6371 failed 7d6211b
💚 Build #6368 succeeded d843e20

cc @mrodm

mrodm · 2025-10-21T16:04:57Z

cmd/export_dashboards.go


 Use this command to download selected dashboards and other associated saved objects from Kibana. This command adjusts the downloaded saved objects according to package naming conventions (prefixes, unique IDs) and writes them locally into folders corresponding to saved object types (dashboard, visualization, map, etc.).`

+	newDashboardOption = "Working on a new dashboard (show all available dashboards)"


Title for the option to be selected in order to show all the dashboards from the Serverless instance.

This option is shown the first one in the list.
Example:

$ elastic-package -C ../integrations-main/packages/nginx export dashboards Export Kibana dashboards Which packages would you like to export dashboards from? (nginx-2.3.2) > Working on a new dashboard (show all available dashboards) elastic_package_registry-0.3.1 fleet_server-1.6.0 nginx-2.3.2 synthetics-1.4.2

mrodm · 2025-10-21T16:06:45Z

cmd/edit.go

+		// Not mandatory to get the package name here, but it would be helpful for users
+		// to select by default the package where they are located if any.
+		defaultPackage := ""
+		packageRoot, err := packages.MustFindPackageRoot()
+		if err == nil {
+			m, err := packages.ReadPackageManifestFromPackageRoot(packageRoot)
+			if err != nil {
+				return fmt.Errorf("reading package manifest failed (path: %s): %w", packageRoot, err)
+			}
+			defaultPackage = m.Name
+		}


I think it could be helpful for the user to select the package where they are located if any. But this should not raise an error if the command does not run from a package.

mrodm added 4 commits October 17, 2025 19:08

Export dashboards using EPM API endpoints

dc54003

Just execute the new method on Serverless

2fa7ec3

Select package from current directory as default

66a1770

Use same method to find dashboards in Serverless via export API

54be65a

Extract function to select dashboard IDs

mrodm self-assigned this Oct 20, 2025

mrodm commented Oct 20, 2025

View reviewed changes

mrodm added 2 commits October 20, 2025 17:38

Increase number of packages per page

d843e20

Remove Response field in response - not present

7d6211b

mrodm commented Oct 20, 2025

View reviewed changes

Add comment

d7b269b

mrodm added 6 commits October 21, 2025 10:22

Check success variable in import saved objects response

6542265

Apply same changes for edit command when running with Serverless

a8b59fb

Remove unused parameters and fields

b6dccd2

Re-phrase new dashboard option

a2b0823

Re-order function

53cdeb3

Rename struct to get response from bulk_assets

9cb72c3

mrodm commented Oct 21, 2025

View reviewed changes

mrodm added 2 commits October 21, 2025 16:45

Make private more structs

14db2d7

Delete comment

243539a

mrodm commented Oct 21, 2025

View reviewed changes

mrodm marked this pull request as ready for review October 21, 2025 16:19

mrodm requested a review from a team October 21, 2025 16:19

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Allow to export dashboards from Serverless environments #3007

Allow to export dashboards from Serverless environments #3007

mrodm commented Oct 20, 2025 •

edited

Loading

Uh oh!

mrodm Oct 20, 2025 •

edited

Loading

Uh oh!

mrodm Oct 20, 2025

Uh oh!

mrodm commented Oct 21, 2025

Uh oh!

mrodm Oct 21, 2025

Uh oh!

elasticmachine commented Oct 21, 2025 •

edited

Loading

Uh oh!

mrodm Oct 21, 2025

Uh oh!

mrodm Oct 21, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants


		Use this command to download selected dashboards and other associated saved objects from Kibana. This command adjusts the downloaded saved objects according to package naming conventions (prefixes, unique IDs) and writes them locally into folders corresponding to saved object types (dashboard, visualization, map, etc.).`

		newDashboardOption = "Working on a new dashboard (show all available dashboards)"

Allow to export dashboards from Serverless environments #3007

Are you sure you want to change the base?

Allow to export dashboards from Serverless environments #3007

Conversation

mrodm commented Oct 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Author's checklist

Example of usage

How to test this PR locally

Uh oh!

mrodm Oct 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mrodm Oct 20, 2025

Choose a reason for hiding this comment

Uh oh!

mrodm commented Oct 21, 2025

Uh oh!

mrodm Oct 21, 2025

Choose a reason for hiding this comment

Uh oh!

elasticmachine commented Oct 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

💛 Build succeeded, but was flaky

Failed CI Steps

History

Uh oh!

mrodm Oct 21, 2025

Choose a reason for hiding this comment

Uh oh!

mrodm Oct 21, 2025

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

mrodm commented Oct 20, 2025 •

edited

Loading

mrodm Oct 20, 2025 •

edited

Loading

elasticmachine commented Oct 21, 2025 •

edited

Loading